Detection of systematic errors in quantum experiments 
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When systematic errors are ignored in an experiment, the subsequent analysis of its results be- 
comes questionable. We develop tests to identify systematic errors in experiments where only a 
finite amount of data is recorded and apply these tests to tomographic data taken in an ion-trap 
experiment. We put particular emphasis on quantum state tomography experiments and present 
two detection methods; the first relies on ideas similar to entanglement witnesses or Bell inequalities 
while the second is based on the generalized likelihood ratio test. 
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Introduction. — Measurements are central to acquiring 
information about the underlying system in any quantum 
experiment. However, in particular for quantum systems 
of increased complexity, the analysis of all measurement 
data gets challenging when one deals with both statisti- 
cal and systematic errors. Statistical errors refer to the 
intrinsic problem that true probabilities, as predicted by 
the laws of quantum mechanics, are never accessible in 
any real experiment but are merely approximated from 
count rates which lead to relative frequencies. A well- 
known example where such statistical effects play a dom- 
inant role is in quantum state tomography the task 
to determine a previously unknown state by means of 
appropriate measurements. Here the deviations between 
probabilities and relative frequencies cause severe prob- 
lems in the actual state reconstruction, since naively us- 
ing the frequencies in Horn's rule easily leads to unphys- 
ical "density operators" , meaning that some eigenvalues 
are negative. This problem can be circumvented using re- 
construction principles that explicitly account for these 
statistical effects 0] . 

The analysis is generally more complicated because of 
additional systematic errors, e.g., drifts in the state gen- 
eration, misalignment in the measurements or fluctua- 
tions of external parameters in the setup. In order to re- 
construct the state from the observed data one requires 
an operator assignment for each classical outcome of the 
performed measurements. This measurement model is es- 
sential, not just for quantum state tomography, but also 
to certify state characteristics like entanglement via en- 
tanglement witnesses Q or in applications like quantum 
key distribution to prove security in the calibrated device 
scenario [5|. However, in a real experiment the measured 
observables (and its independent repetitions) might devi- 
ate from this employed description due to some system- 
atic errors. This mismatch can have severe impact on 
the analysis outcome and can lead to, for example, spu- 
rious entanglement detection @ , apparent violation of a 
Bell inequality M or complete insecurity in quantum key 
distribution [8|, |9[ . Though deviations of this kind have 
been discussed and partially counter-measured by differ- 
ent techniques [10l - [l4| . it has not yet been investigated 



how to distinguish them bom statistical errors, 
ception is given in Ref. 
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detected using measurements on subsequent states. 

In this paper we present experimentally and theoret- 
ically two methods to detect whether systematic errors 
are statistically significant, i.e., if there is only a small 
probability that such observed data were generated by 
statistical effects. In this case further analysis becomes 
irrelevant and the data can be rejected directly. Conse- 
quently, we strongly encourage that these tests are ap- 
plied before reconstructing actual quantum states. We 
want to emphasize that these techniques can only falsify 
the assumptions but never serve as a justification that 
systematic errors are absent. Two procedures are pre- 
sented, the first resembles the concept of entanglement 
witnesses [H, [l^l and hence directly makes use of the 
structure of quantum states, while the second is based on 
the generalized likelihood ratio test [l^, [l^ • Let us point 
out that other procedures, like the prediction-based-ratio 
analysis of Ref. [l^l or the chi-square goodness-of-fit 21 1 , 
arc other alternatives here. 

Tomography setting. — A common tomography proto- 
col is the Pauli measurement scheme Q which has been 
used for n qubit systems in ion traps [2^ or photonic 
setups [23|. Here for each of 3" possible combinations 
of Pauli operators on n qubits one locally measures in 
the associated eigenbasis which provides 2" distinct out- 
comes, thus in total 3" x 2" = 6" different outcomes. 
Note that an n qubit density operator is already deter- 
mined by 4" — 1 real parameters, meaning that this mea- 
surement scheme collects largely over-complete data. 

More generally, we consider a tomography protocol 
that performs measurements for different settings la- 
belled by s and registers the respective frequencies = 
ml/Ns, where m| denotes the counts of the specific out- 
come k in Ns repetitions of this experiment. The repeti- 
tions TVs are assumed to be equal for each setting. The 
attributed measurement operators to these classical data 
arc denoted by and have the property that they span 
the complete operator space in order to enable a full re- 
construction of the unknown density operator. Most of- 
ten, as is the case with the Pauli measurement scheme. 
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this set is over-complete, i.e., the operators contain Un- 
ear dependencies which can be expressed as linear iden- 
tities ~ using real coefficients c|. The set of 
probabilities consistent with this quantum tomography 
model are all distributions Pqm(fc|s) = tr(pA/|) that can 
be written using a density operator p. 

Witness test. — The set of distributions consistent with 
the assumed quantum model can be characterized by lin- 
ear inequalities. This is in analogy to entanglement wit- 
nesses 0, [13 for separable states or Bell inequalities [l^l 
for local hidden variable models. Consider a set of real 
coefficients w = wf. that define a positive semidefinite 
operator via Yl^k^k = Z^j ^ 0, i.e., all eigenvalues are 
non-negative. Then the expectation value of any proba- 
bility distribution from the quantum model Pq,n satisfies 
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^w?Pqm(fck)=tr(pZ^) >0. (1) 



Thus a distribution P with w • P < is incompatible with 
the assumed quantum model, and any such distribution 
can indeed be detected by a witness of the described form 
(even with partial information (25j). 

Equation ([T]) is formulated on the level of probabili- 
ties which are not accessible in the experiment. Never- 
theless one can replace the probabilities by the observed 
frequencies / = and consider the sample mean value 
w ■ f = J2'^kfk of witness. Then w ■ f > does 
not need to hold anymore because statistical effects can 
produce a negative value. However, the probability to ob- 
serve large deviations from the true mean is bounded and 
decreases exponentially with the number of performed 
repetitions. A quantitative statement is given by HoefFd- 
ing's tail inequality j26j , as similarly used for example 
in efficient fidelity estimation [27, .28| . Let us empha- 
size that this inequality is even valid for small data sets 
containing only few or no counts for certain outcomes. 

Proposition 1. — Consider a witness w = wf. obeying 
^w|M| = Zui h 0- If the data are generated by the 
quantum model Pqm(fc|s) = tr(pM^), then for all t > 0, 



Pvoh[w ■f<-t]< cxp{-2t^N,/Cl) 



(2) 



with Cl = J2s (Wmax - <un) : ^hcrc w^^^/^^j^ are the 
optima for setting s over all outcomes k. A proof is given 
in the appendix. 

The interpretation of this is as follows: Suppose that 
one carries out an experiment for an arbitrary but pre- 
viously chosen witness w and fixed error probability 
a, that one still tolerates before one announces a sys- 
tematic error, maybe a — 1%. Using Proposition 1 
one can assign a minimal necessary violation given by 
ta = \J —C'w log(a)/2A's. If one now registers frequencies 
/obs that beat this value w ■ /obs < —ta-, then the prob- 
ability that these data have originated from a quantum 
model by chance is less than a and one says that a sys- 
tematic error is significant (at significance level a). Since 
error values a are often a matter of choice one states re- 
sults in terms of the smallest error probability that would 
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FIG. 1. (Color online) The admissible probabilities for the as- 
sumed quantum model Pq,n form a convex, lower dimensional 
subset within all possible probability distributions (dashed 
cube). This depicted dimension reduction stems from the ad- 
ditional linear relations that a probability distribution from 
the quantum model must fulfil. The linear relations are 
checked by witnesses iul. In contrast the witnesses lup verify 
the positivity of the underlying density operator. 



still allowed detection, hence exp[— 2(w • /obs)^A"s/Cj] if 

W ■ fobs < 0. 

Witness structure. — Each above defined witness w can 
be decomposed into two conceptually different parts. 
One that solely verifies positivity of an underlying den- 
sity operator, denoted as wp in the following, and into 
another part iul that only checks the linear dependencies, 
such that one obtains in total w = wp + u'l ■ It turns out 
that these two parts of the witness are orthogonal. Let 
us point out that the witness wp uniquely describes the 
operator J2'^Pk-^k ~ ^w, while the witness wl (and 



because of the linear 



also — wl) vanishes 
dependencies. Figure [T] gives a schematic picture of this 
situation. 

Issue of negative eigenvalues. — The above framework 
provides an answer to the issue of negative eigenvalues 
in linear inversion, since it is connected to witnesses of 
the type wp. Linear inversion refers to the state recon- 
struction process in which one estimates the unknown 
density operator of the system by using the observed fre- 
quencies in Born's rule tr(Af^p) = Since this set of 
linear equations is typically not exactly solvable because 
of over-complctcncss one selects the operator pis which 
minimizes the least squares, J^ifk ~ ti'(pisA/f )]^. Since 
one ignores the positivity constraint this operator pis will 
often represent an invalid density operator because some 
eigenvalues are negative, i.e., {ip\p\s\ip) < 0. The fol- 
lowing statement provides a bound on the probability of 
negative expectation values. 

Proposition 2. — Let p\s be the linear inversion using 
least squares and consider a given vector \^). If the 
data are generated by the quantum model Pq„i(A:|s) = 
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tr(pM^), then for all t > 0, 

Prob[(^|pi,|i/;) < -t] < cxp{~2t^Ns/Cl) (3) 

with C,^ as given in Proposition 1 computed from the 
unique wp satisfying X^'^^pfc^'^fe — IV') (^1- A proof is 
given in the appendix. 

The interpretation of this result is as follows: The 
probability to successfully guess in advance a state lip) 
where pis has a negative expectation value is exponen- 
tially suppressed. 

Likelihood ratio test. — In addition to the attributed 
quantum model Pqni{k\s) = tr(pM^) we can also describe 
the observations with a more general model assumption 
of independent distributions Pind{k\s) = p| > and 
SfePfe = 1 fo'^ each setting s. The question whether the 
observed data arc compatible with the assumed quantum 
model can now be addressed by comparing the maximal 
likelihoods of either model [l^ . 

For that, we start from the likelihood for a distribution 
P given the observed data /, which then is 

LiP) = Uk,sPim'''f^, (4) 

ignoring the multinomial prefactor. A quantum state pmi 
that maximizes the likelihood L{P) is considered to be a 
good estimate for the physical state [H, 0|- In contrast, 
for the model with all independent distributions, the op- 
timum is given by p^. = f^. Since the quantum model 
is contained in this more general model, the likelihood 
of any quantum model can at best be equal to this opti- 
mal likelihood. Thus one finds L{f) > L[ti-{p„^\AI^)] or 
equivalently, a non-negative log-likelihood ratio Aqm = 
21ogL(/)-21ogL[tr(p,„iA4«)]. 

The likelihood ratio test is based on the observation, 
that if the data are indeed generated from the quantum 
model then the probability for outcomes which satisfy 
Aqm > t decreases rapidly if the value t exceeds a certain 
value. While in general the distribution of Aqm is cum- 
bersome to access, we here can transform our situation 
to a canonical structure, for which it is known that such 
a ratio is distributed according to a chi-square distribu- 
tion [2^. This statement known as Wilks' theorem [s^l 
can be applied directly if one performs the optimiza- 
tion (rather than over quantum models) over probabil- 
ities Pnqni{k\s) = tr{X M^) that can be written in terms 
of a Hermitian operator X. Note that X can have neg- 
ative eigenvalues, indicated by the subscript "n", while 
still obeying the positivity constraints ti{XM^) > for 
the measurements Af|. With Xmi being a corresponding 
optimum we now study the log-likelihood ratio 

Anqm = 21ogL(/) - 21ogL[tr(XmiA4f)]. (5) 

Proposition 3. — If the data are generated by the d- 
dimensional quantum model Pqmik\s) = tr(pM|) with 
K outcomes for each of the S settings, then for all t > 0, 
as Ng — >■ 00, 

Prob[Anqm ><] ^Q(A/2,t/2), (6) 



with the dimension deficit A = {K — l)^ — (d^ — 1) and 
the regularized incomplete gamma function Q |3l| . A 
proof is given in the appendix. 

The interpretation and application of this result is 
analogous to Proposition 1. Though Proposition 3 is 
only a strict statement in the asymptotic case A^g — > 00, 
Eq. ^ gives reliable values already for moderately large 
A"s, as we will demonstrate below. 

Experimental setup — Experimentally, we study tomo- 
graphic data from an ion-trap quantum processor. Our 
system consists of a string of '"^Ca"'" ions confined in a 
linear Paul trap where each ion represents a qubit. The 
quantum information is encoded in the Si/2(m=— 1/2) 
ground state |1) and the metastable D5/2(ni=— 1/2) state 
|0). More details on the experimental parameters can 
be found in Ref. [s^ . Qubit manipulation is performed 
by a series of laser pulses, adjusted by frequency, du- 
ration, intensity, and phase. Collective operations are 
of the form exp{-i9S(j,/2) with S4, = J2?=i'^4>,ii '^<t',i = 
cos{(l))ax,i + sin(0)(Ty^; and a.^i the according Pauli op- 
erator acting on ion I. The rotation angle 6 = Ht is 
defined by the global Rabi frequency fl of all qubits and 
the duration r of the laser pulse. Additionally, single ions 
can be addressed with a far off-resonantly detuned beam. 
Here the induced AC-Stark effect induces a rotation of 
the form exp{—i9az,i/2) on ion Z, with 9 determined by 
the detuning, intensity and pulse duration. The combi- 
nation of single-qubit phase rotations and collective S'^ 
rotations allows us to implement any local operation on 
the entire quantum register and, subsequently, to per- 
form state tomography on a given state according to the 
Pauli measurement scheme. 

The presented treatment of the implemented opera- 
tions assumes homogeneous illumination via the global 
beam, i.e., an identical Rabi frequency for all ions, as 
well as a narrow beam for individual phase rotations that 
solely illuminates the addressed ion. In an experimental 
realization, the finite width of the focused beam results 
in residual ion-light interaction on next-neighbor qubits. 
The Rabi frequency of ion k when addressing ion j can 
be described by the addressing matrix Qj_k- Thus the 
operation on the qubit register can then be written as 

cxp(-i^ilj_fcTcr^,fc/2). (7) 

fc 

The addressing quality can be quantified with a cross- 
talk parameter e = maxj:^k(S^j,k/^j,j), which can be in- 
creased by defocusing the addressed laser beam. 

Using this setup we perform tomography on various 
kinds of states and investigate whether the obtained 
data suffer from any kind of systematic errors. This in- 
cludes data for Greenberger-Horne-Zeilinger states on 4 
ions, \GHZ) = (|0000) + \nn))/V2, where we inten- 
tionally increased the cross-talk e in order to test the 
presented techniques, a large data set on a two-qubit 
Bell state \tp~) — (|01) — |10))/-\/2 and measurements 
on the ground state \SSSS) = |1111). Moreover we re- 
analyse observations on a W-state on 5 qubits, \W) = 
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TABLE L Analysis of experimental data. We use the data 
from 3 experiments preparing a GHZ-state on n = 4 qubits 
with different cross-talk parameter e, data from a Bell-state 
on two qubits, the ground (SSSS) and a bound entangled state 
(BE) on 4 qubits and a W-state on 5 qubits. The number of 
events per measurement setting is shown in the column A'^s. 
The columns wl and wp show the lowest error probabilities of 
the test given by Proposition 1, checking linear dependencies 
or positivity respectively, which would have allowed a system- 
atic error detection, "— " stands for no detection. The column 
LR lists similar values according to Proposition 3, while LR* 
is obtained using a parametric bootstrapping method [IJ] 
with a 1000 samples. The bracketed value suffers from an 
insufficient number of samples. For data marked with * we 
manually increased the cross-talk, while ^ could have been 
intensity fluctuations. 



(100001) + 100010) + 100100) + 101000) + |10000))/V5 and 
a bound-entangled Smolin state (33| . 

Empirical findings. — At first we implement the wit- 
ness test, see Table HI Let us stress that Proposition 1 
does not allow us to determine and to evaluate the wit- 
ness w from the same data. If one would do so then 
one efi^ectively evaluates miuu, w ■ f instead of w • / as 
required in Proposition 1. Because of that we divide 
the observed data into two equally sized parts, yielding 
frequencies /i and /2. Afterwards we use the first part 
/i to determine a reasonable witness w, which is finally 
evaluated on the second data, w ■ f2- Here we choose 
either of the two types of witnesses testing positivity wp 
or linear dependencies ujl- As witness wp we select the 
witness that corresponds to the projector onto the small- 
est eigenvalue on the linear inversion pis using the first 
data set /i. For the linear dependencies we use the wit- 
ness wl = —fi + tr(yOisM^), because it gives the largest 
negative expectation value wl • fi on the first data. Let 
us point out, that the employed choices here might not 



be optimal [35|. If the observed value w • /2 is negative, 
we ask for the statistical significance as explained after 
Proposition 1, i.e., we state the lowest error probability 
that would have still allowed detection via this witness 
test, exp[-2(w • f2)'^Ns/C^]. Values as low as 10^^% are 
clearly unacceptable. 

For our data, the witness wp reliably detects the arti- 
ficially introduced cross talk for the GHZ-state with pa- 
rameters e = 12% and e — 20%. In contrast, the witness 
Wl turns out to be less powerful for these examples. 

The likelihood ratio test, which we investigate as sec- 
ond method, is best suited for a larger number of sam- 




FIG. 2. (Color online) Distribution of the log-likelihood ra- 
tio ([5]) from a 411-fold repetition of a Bell-state tomogra- 
phy experiment. We show the fraction of runs having a log- 
likelihood ratio Anqm larger or equal to the specified value t. 
In the upper graph, the shaky blue line corresponds to the 
experimental data, while the smooth green line is the predic- 
tion according to Wilks' theorem (|6}. The lower graph shows 
the difference between both curves. A simulation of the ex- 
periment yielded a similar result (not shown). 



pies, since Proposition 3 makes strictly speaking only a 
statement for TVs — )■ (X). In Figure [5] we compare the em- 
pirical distribution between a two-qubit Bell experiment 
using 150 samples/setting and the predicted distribution 
according to Wilks' theorem. Hence for the two-qubit 
case this number might already be sufficiently close to 
this limit. This observation can be further supported 
via a comparison with a bootstrapping method [3J] (see 
appendix) which mostly produces similar results as the 
ones obtained from Proposition 3. Based on these obser- 
vations we are confident that the results using Proposi- 
tion 3 for finite Ng are trustworthy for all data from Ta- 
bleUexcept for the W-state, which has a too low number 
of samples. Evaluating the experimental data we detect 
again the manually increased cross talk in the GHZ ex- 
periments, but now also some discrepancies in the SSSS 
experiment, which most likely occurred because of inten- 
sity fluctuations during the experiment (36j . 

Conclusion & Outlook. — The tomographic reconstruc- 
tion of a quantum state can be problematic since non- 
physical properties, such as negative eigenvalues, might 
occur. One possible way out is to use reconstruction 
schemes, which by construction result in a valid state. 
Then, however, serious concerns remain, as the negative 
eigenvalues can be a signature of systematic errors. We 
have provided tests which can be used to distinguish sys- 
tematic from statistical errors in quantum experiments. 
These tests were shown to recognize systematic errors in 
real tomographic data from ion trap experiments. 

We formulated our result for the important case of 
state tomography, however, our methods can be applied 
to check other assumptions like the non-signaling condi- 
tion in Bell experiments. From the more general perspec- 
tive, many experiments in physics aim at determining 
the parameters in an assumed theoretical model. Our re- 
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suits show that it is possible to give rigorous estimates on 
whether the assumed model class is indeed appropriate 
or not. 
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APPENDIX 

Proof of Proposition 1 . — The proposition uses HoefFd- 
ing's tail inequality |26j and the property that valid quan- 



tum distributions Pqm have a non-negative expectation 
value w ■ f > due to Eq. dT]). Hoeffding's inequality 
states that the sample mean X = '^Xi/N of N inde- 
pendent, not necessarily identical distributed, bounded 
random variables Xi with Prob[A'i G [a,, 6,]] ~ 1 for 
i = 1, . . . N satisfies 

Prob[X-E(X) < -t] < cxp[-2Nh^/'^{b,~a,)^] (8) 



6 



for a\\ t > and W,{X) denoting the mean value of X. 
In order to prove the proposition we identify X with the 
sample mean of the witness. This is achieved as follows: 

Suppose that Yf denotes the random variable associ- 
ated with the i-th repetition of the measurement set- 
ting s. In case of the measurement outcome k, Yf 
takes the value Sw^ where S denotes the total num- 
ber of measurement settings. It is bounded between 
Yf e [S'w^jjjj, S'w^g^x]. Then the sample mean of all these 
variables 



one obtains 



fp = wp ■ /, 



Pfc 



w ■ 



(14) 
(15) 



Y 



(9) 



yields values 



together with the solution of least squares tr(M^/9is) = 
/p|- 

Proof of Proposition 3. — We start by a rough state- 
ment of Wilks' theorem (cf. e.g., 13.8.1 in Ref. [30|). Sup- 
pose that we have a family of models, that is sufficiently 
smoothly parameterized by an open set C M'^ and let 
A be a subspace of W of dimension r — A. If we draw TV' 
samples from our model with some (unknown) parame- 
ter z G Anil, then the distribution of the log- likelihood 
ratio 

Aa = 21ogsupL(a;) — 21og sup L{y) (16) 



xen 



yeAnfi 



Wi.rrii 



J2 '^kPk ^w-f. 



(10) 



converges to the Xa distribution as A^' oo. Hence, 



where denotes the counts of the specific outcome k 
in Ns repetitions of the measurement settings s. 

Using Hoeffding's inequality together with the prop- 



N 



lim Prob[AA > i] = Q(A/2,i/2). 



(17) 



For our model, the set of parameters O is given by p| 
with k = 1, . . . ,K — 1 obeying > and X^fePfc < 1- 
Furthermore, we let A be the subspace where qf. ~ 
erty that lE(y) = w-Pqm > holds for any valid quantum tr(XM^) for some Hermitian matrix X. We now basi- 



distribution Pqm due to Eq. ([!]) one arrives at 

Prob[u; •/<-*] = Prob[y < -t] (11) 

< Prob[y - E(y) < -t] (12) 

< exp[-2<27V,/C,2], (13) 

with = X]s('^max ~ ^min)^- ^hc first inequality holds 
because the set of all outcomes satisfying Y < —t \s & 
subset of y — lE(y) < —t. This concludes the proof of 
the proposition. 

Proof of Proposition 2. — In order to prove the proposi- 
tion we show (V' I Pis IV') = w-p ■ f , which can then be used 
in Proposition 1 to obtain the final statement. 

Given a valid decomposition w satisfying ^ wf.M^ = 



cally arrived at Proposition 3. It only remains to change 
the notation from to pf. via = 1 — J^kPk 
to admit a more relaxed notation by essentially writing 
max^glY instead of sup^^j^. 

Bootstrapping for Table\^ — We first calculate the max- 
imum likelihood state p^i from the measured data and 
then simulate the tomographic process using that state. 
From the simulated data we then calculate the log- 
likelihood ratio Anqm- We repeat this procedure 1000 
times and compare the distribution with the predicted 
chi-square distribution. In our examples, the distribution 
matches always almost perfectly, however with a slightly 
different parameter A. We then determine A' to be such, 
that (5(A'/2,m) = 1/2, where m is the median of the 
sampled distribution. Using A', wc obtain the values of 
the column LR* of Tabic HI 



